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Smart Start is a multi-disciplinary, comprehensive, 
community-based initiative to serve North Carolina children under age 6 and 
their families to ensure that all children enter school healthy and prepared 
to succeed. To achieve this, local county partnerships have focused both 
their attention and their funds on implementing child care programs, family 
programs, and health services. Evaluation strategies included measuring 
changes in child care quality, family functioning, and children's receipt of 
health services, and the long-term outcome of school success. This report 
highlights four main program goals: (1) children are healthy and prepared to 

succeed in school; (2) families effectively fulfill their role as primary 
providers, nurturers, and teachers; (3) high quality, affordable services for 
children and families will be available; and (4) North Carolina counties 
value children and families by providing options and resources, and 
encouraging collaboration. The report presents the main data collection 
strategies being used to evaluate progress for each goal, followed by brief 
summaries of recent results . The most significant findings to date are that 
child care quality improved as a result of Smart Start and that children from 
low-income families who attended a Smart Start -involved child care program 
were significantly more prepared for school than were similar children who 
attended other centers, or no centers at all. Speech problems were the most 
likely developmental problem to receive preschool treatment. Playground 
improvement grants did result in improved playground safety. Also noted was 
increased collaboration among individuals and agencies involved with young 
children and families. An appendix contains a list of reports from the Smart 
Start evaluation team. (KB) 
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North Carolina’s Smart Start Initiative: 

1998 Annual Evaluation Report (January. 1999) 

This report summarizes the evaluation findings 
to date from both quantitative and qualitative data sources. 
The data were gathered as part of the Smart Start evalua- 
tion contract from the NC Division of Child Development to 
the University of North Carolina at Chapel Hill. We want to 
thank all child care directors and providers, service agency 
staff, and other Smart Start participants who have helped 
with various aspects of the evaluation. 



For additional copies of this report or other Smart Start 
evaluation reports contact Marie Butts at: 

The Frank Porter Graham Child Development Center 
Campus Box 8180, UNC-CH 
105 Smith level Road 
Chapel Hill. NC 27599-8180 

(919) 966-4295 

e-mail: butts@unc.edu 



500 copies of this public document were 
printed at a cost of $597.00 or $1.19 per copy. 



1998 

Annual 

Evaluation 

Report 



North Carolina’s Smart Start Initiative 

Smart Start is a multi-disciplinary, comprehensive, community- 
based initiative to serve North Carolina children under age 6 and 
their families. The major long-range goal of Smart Start is to 
ensure that all children enter school healthy and prepared to 
succeed. To achieve this goal, local county partnerships have 
focused both their attention and their funds on three major areas 
of service implementation: child care, family programs, and 
health services. The evaluation team considers changes in these 
areas to be intermediate-term outcomes, that is, changes that 
should lead to the longer-range goal of increased preparedness 
for school. Therefore, the evaluation data collection strategies 
have included measuring changes in child care quality, family 
functioning, and children’s receipt of health services, as well as 
the long-term outcome of school success. 



These highlights are organized by the four main goals 
of Smart Start. The results included here have been presented 
in separate and more detailed reports which 
are listed at the end of this document. 




1 



Smart Start 

Goal 1 

Children are healthy 
and prepared to succeed 
in school. 

North Carolina public schoois obtain 
heaith and "readiness" data on 
entering kindergartners in a variety 
of different ways, because no 
uniform database exists to monitor 
chiidren 's progress across counties. 
The evaiuation team has used the 
Kindergarten Teacher Checklist with 
iarge sampies of randomiy seiected 
chiidren and smaiier sampies of 
purposefuiiy seiected chiidren to 
measure chiidren's cognitive, 
ianguage, motor, and sociai skiiis. 
Data coiiectors have aiso coded 
immunization, screening, and other 
heaith data from thousands of 
Kindergarten Health Assessment 
forms. Severai findings about heaith 
and preparedness for schooi foiiow. 



The large random samples show that NC children are entering 
kindergarten with a wide range of skills, but about 18% of them 
are judged by their teachers as not ready to participate success- 
fully in school. Mean ratings were the same in 1995 and 1997 
and were also about the same for partnerships that had been 
involved in Smart Start since the beginning and for partnerships 
that only recently become involved. In these large samples we 
do not know which children received Smart Start-funded ser- 
vices and which did not. 

( For more results, see Kindergartners' Skills in Smart Start 
Counties in 1995.) 

In a single-county study, we have been able to specifically 

identify children who have attended child care programs that 

have participated in Smart Start quality improvement efforts 

and compare them to children who have not. The kindergarten 

entry skills of children from low-income families who have 

attended Smart Start-involved child care centers were rated 

statistically significantly higher than children who had not 

attended such centers. Children from middle-income families 

were rated significantly higher if they had attended any center- 

based child care before kindergarten compared to children who 

had not attended child care at all. 

( For more details, see The Effects of Smart Start Child Care 
on Kindergarten Entry Skills.) 
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Kindergartners who had attended child care had more skills 
than those who did not attend child care, which is consistent 
with other research demonstrating that child care can promote 
cognitive, language, and social skills. 

Children from poor families had fewer skills than those from 
non-poor families, supporting long-standing research that 
poverty is a component of school failure. These findings also 
support Smart Start’s focus on getting more children from 
poor families into child care and improving the overall quality 
of child care. 

Most kindergartners were fully immunized once they entered 
kindergarten, but only 53.3% had been immunized on time. 
Many children apparently are given their last immunizations 
right before school starts, but these children should have 
received the immunizations earlier. 

Of entering kindergartners, 2% failed a hearing test, 7% failed 
a vision test, and 25% had at least one identified health prob- 
lem. These are the kinds of health needs that Smart Start 
programs are attempting to find and treat earlier than 
kindergarten entry. The evaluation team will monitor these 
types of health needs over time to note possible changes. 



Speech problems seem the most likely 
developmental problem to receive 
treatment in the preschool years, with 
over 40% of entering kindergartners 
who were noted to have a speech 
problem having already been referred 
to a Developmental Evaluation Center 
(DEC). Child care centers, health 
providers, and parents are more likely 
to perceive the need for speech inter- 
vention compared to other problems, so 
more children with speech difficulties 
have received treatment before they 
enter school. Even so, almost 60% of 
children with speech problems had not 
yet been referred. 
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Smart Start is reaching the broad range of families it intended 



Smart Start 

GoqI 2 

Families effectively fulfill 
their role as primary 
providers, nurturers 
and teachers. 

In 1995, the evaluation team inter- 
viewed 356 families who had received 
some type of child care, health, or 
family support service from Smart Start. 
Families were randomly selected. The 
interviews covered child care, health, 
and community services, and included 
some questions used in national surveys 
so as to compare NC Smart Start 
families to other samples. 



to reach, serving the unemployed, the working poor, and the 
middle-class. Sixty-three percent (63%) of low-income and 
90% of middle-income Smart Start families in our sample 
were employed. 

Families participating in Smart Start engaged in educationally 
important activities with their child (such as reading, playing 
number games, telling stories, singing together) as often, if not 
more often, than a national sample of parents of preschoolers. 
The similarity between the Smart Start and national sample is 
even more striking because of the larger proportion of low- 
income parents included in the Smart Start sample. 

A high percentage of Smart Start families (79%) reported that 
they have the strength to cope with the pressures on them. 



although this percentage was slightly lower than found in a 



random sample of NC families (84%). 



Participation in religious activities was high among Smart Start 
families (60%) and for many families was the only involvement 
in community groups reported; 25% of Smart Start families 
were not involved in any community group. 
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Parents learn about services for their children and themselves 
through word of mouth, friends and relatives, and the phone 
book. An activity frequently funded by Smart Start — creating a 
directory of community services — was ranked fourth. These 
findings indicate that there is a continuing need for distribution 
of information about child and family services to a wide variety 
of agencies and people, not just to professionals or through 
service agencies. Especially given the high percentage of fami- 
lies involved with church or religious organizations, providing 
information through these groups is a way to reach a large 
number of people. 

Almost one-fourth of families interviewed said they needed 
parenting education programs. 



Smart Start is funding a wide variety of 
famiiy-focused programs. Most of them 
appear to be desired by a iarge number 
of families, and many are serving the 
need for information, education, and 
training as expressed by many parents. 



( For further detaiis, see Families 
and the NC Smart Start Initiative.^ 



Low-income families were more than twice as likely to report 
one or more barriers to obtaining needed services, including cost 
of services, inconvenient hours, lack of child care, and negative 
attitudes of the staff at service agencies. 
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Smart Start 

Goal 3 

High quality, affordable 
services for children and 
families will be available. 



In pioneer Smart Start partnerships, the quality of child care 
was significantly greater in 1996 than in 1994, both as mea- 
sured by the percentage of AA centers and by observations of 
quality in preschool classrooms. From 1994-1996 the percent- 
age of classes rated as “good” or above on the ECERS increased 
from 14% to 25%. 



Data on the quality of NC child care 
have been gathered through preschooi 
ciassroom observations (using The 
Early Childhood Environment Rating 
Scale, ECERS j and director interviews 
in iarge sampies of chiid care centers 
in 1994-95 and 1996-97. Data on 
playground safety were coiiected 
from 34 playgrounds in two counties 
in 1997. 



These quality changes appear to be related to Smart Start 
participation as evidenced by four specific findings. 

1) The proportion of funds a partnership spent on child care 
quality improvements significantly predicted quality, with 
more funds spent on child care quality improvement 
positively related to ECERS quality. 

2) The percentage of full-funding received by a partnership 
affected the influence of proportion of funds spent on child 
care. Proportion spent on child care was not as strongly 
associated with quality in counties that received a low 
percentage of their full-funding amount as it was in 
counties that received a high percentage of their full- 
funding amount. 

3) The number of child care quality improvement activities 
that an individual center participated in was significantly 
related to their ECERS quality score — more activities 
were associated with better quality. 

4) The average quality of the 91 centers that were visited in 
both 1994 and 1996 increased significantly, including an 
increase in the proportion of centers licensed at the 
higher AA level. These findings all support the conclusion 
that improved child care quality is associated with Smart 
Start participation. 



( For more detail, see Effects of Smart Start on Preschooi 
Chiid Care Quaiity.) 
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The percentage of AA-licensed child care centers (a higher level 
of quality care than the A license) is increasing in many counties 
in NC. but it is increasing at a faster rate in Smart Start counties 
than in non-Smart Start counties. 

The quality of care in centers that included children with dis- 
abilities was higher than that found in programs enrolling only 
nondisabled children, suggesting that high quality child care is 
available for children with disabilities and their families. 

More than half of all families interviewed expressed a need for 
financial assistance for child care and most were receiving it. 
Half of low-income families spent more than 10% of their 
income on child care. 

The safety ratings of child care center playgrounds that partici- 
pated in playground improvement programs sponsored by Smart 
Start were significantly higher for 1 1 of 15 safety criteria than 
playgrounds that had not been part of an improvement program. 
Previous research shows that this should result in fewer injuries 
to children. 

(For more detail, see Effect of a Smart Start Playground 
Improvement Grant on Child Care Playground Hazards.) 



In summary, the most important finding 
in the area of child care has been the 
significant improvement in preschool 
classroom quality. The difference is 
significant at a level that scientists term 
“moderate, " that is. a change that is 
meaningful and worthwhile. Finding a 
difference of this magnitude is particu- 
larly notable since, in Smart Start's first 
years, it has been a diverse set of 
“treatments" implemented in a wide 
variety of settings with a varying degree 
of intensity. However, the mean level of 
quality even in 1996 was below that 
which one would hope to achieve 
eventually, so there is still room for 
improvement. The evaluation team has 
just begun a third round of observations 
in over 200 child care programs and will 
report on these data in the fall of 1 999. 
Another important finding is that 
playground improvement grants do 
result in improved playground safety. 
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Smart Start 

Goal 4 

NC counties value children 
and families by providing 
options and resources, and 
encouraging collaboration. 



Data about collaboration and commu- 
nity involvement within partnerships 
have been gathered through qualita- 
tive studies using interviews, focus 
groups, and document reviews with 
various partnership members. 
Because the process of implementing 
Smart Start is so unique and has 
been noted as a model of a new 
approach to community initiatives, 
the evaluation team has each year 
included a qualitative research 
component to investigate the process 
of board and agency collaboration. 
Some of the key findings follow. 



Smart Start appears to have improved local inter-agency 
collaboration among organizations that serve young children 
and their families. The local partnership boards and planning 
process both encourage inter-agency collaboration. However, 
substantial variations across partnerships exist in the levels 
of interagency collaboration. 

( For more details, see Smart Start and Local Inter- 
Organ iza tional Collabora tion . ) 

All pioneer partnerships included children with disabilities and 
their families in their plans and allocated up to 10% of their 
funds for programs serving children with disabilities and their 
families. Inclusion is a strongly supported belief among 
partnership board members and service providers. 

(For more details, see Effects of Smart Start on Young 
Children with Disabilities and their Families.) 

Across partnerships, agencies have increased their collabora- 
tion and cooperation in developing needed resources for young 
children and their families as noted by key participants. 

Many participants feel that this collaboration is one of the 
most striking successes of the initiative. 

( For more details, see Keeping the Vision in Front of You: 

Results from Smart Start Key Participant Interviews.) 
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Local partnership board members fear that politicization of 
Smart Start at the state level has created a defensive mentality 
among partnership boards that somewhat stifles innovation. 
Local board members believe that over the years there has been 
a retraction of local autonomy and increased pressure for more 
standardization and regulation of local partnerships. 

(For more details, see Reinventing Government? 

Perspectives on the Smart Start Implementation Process.) 

Parent and business involvement in the local partnerships is 
essential for partnerships to fully involve their communities, 
but this is a continuing challenge for partnerships. Executive 
directors have assumed the major responsibility for supporting 
parent and business involvement. Partnerships that have a 
higher parent and business involvement rely on multiple strate- 
gies, including recruiting key community leaders, offering a wide 
range of roles to participants, and supporting participation. 

(For more details, see Bringing the Community Into the Process: 
Issues and Promising Promising Practices for Involving Parents 
and Business in Local Smart Start Partnerships.) 



Through interviews and focus groups, 
the evaluation team has noted increased 
collaboration among individuals and 
agencies involved with young children 
and families. The coming together has 
not always been smooth and easy, but 
the long-term goal of improved child 
and family well-being has encouraged 
many already-motivated individuals 
in North Carolina to share their ideas, 
information, and energies towards 
creating more and better programs 
in their own community. 



Sharing information about clients is a form of collaboration. 

The evaluation team conducted a feasibility study of establishing 
a unique identifier client information system in 4 partnerships 
and discovered that several factors, including time, trust, 
funding, and technical expertise were significant barriers 
to creating such a system. This issue is broader than 
Smart Start and needs to be addressed at the state level by 
all service systems. 

( For more details, see Smart Start Client Information System 
Feasibility Study.) 
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Future 

Work 



The findings summarized in this report are measurable, interim 
effects of Smart Start; that is, they are steps along the way to 
improved health and preparedness of children entering kinder- 
garten. This ultimate goal is one that may be accomplished if the 
Smart Start efforts achieve their intermediate goals-better child 
care, improved well-being of families, and greater health re- 
sources for children. Two very significant findings to date are 
that child care quality has indeed improved as a result of 
Smart Start and that children from low-income families who 
attended a Smart Start-involved child care program are signifi- 
cantly more prepared for school than are similar children who 
attended other centers, or no centers at all. We are in the midst 
of conducting studies or analyzing data on family child care 
quality, families receiving subsidies, public-private partnerships, 
and interagency collaboration. These results will be included in 
future reports in 1999. 



For more detailed information about the Smart Start Evaluation, 
please contact Dr. Donna Bryant at (919) 966-4295. 
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The Smart Start Evaluation Team consists 
of several researchers and faculty members 
at the University of North Carolina at 
Chapel Hill. 



Frank Porter Graham Child Development Center 

Donna Bryant. Ph.D., Principal Investigator 
Kelly Maxwell, Ph.D., Project Director 
Kathleen Bernier, Ph.D. 

Ellen Peisner-Feinberg, Ph.D. 

Shari Miller-Johnson, Ph.D. 



Maternal and Child Health Department 
School of Public Health 

Jonathan Kotch, M.D. 



Jordan Institute for Families 
Sc hool of Social Work 

Dennis Orthner. Ph.D. 
George Cole, Ph.D. 



School of Education 

George Noblit, Ph.D. 
Jean Patterson, Ph.D. 
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Appendix 

Reports From the UNC 
Smart Start Evaluation Team 



Emerging Themes and Lessons Learned: 
The First Year of Smart Start 
(August 1994) 

This report describes the first-year 
planning process of the pioneer partner- 
ships and makes some recommendations 
for improving the process. 

Smart Start Evaluation Plan 
(September 1994) 

This report describes our comprehensive 
evaluation plan, designed to capture the 
breadth of programs implemented across 
the Smart Start partnerships and the 
extent of possible changes that might 
result from Smart Start efforts. 

Keeping the Vision in Front of You: 

Results from Smart Start Key Participant 
Interviews (May 1995) 

This report documents the process as 
pioneer partnerships completed their 
planning year and moved into implementa- 
tion. 



Bringing the Community into the Process: Issues and Promising 
Practices for Involving Parents and Business in Local Smart Start 
Partnerships (April 1997) 

This report describes findings from interviews and case studies 
about the involvement of parents and business leaders in the 
Smart Start decision-making process. 

The Effects of Smart Start on the Quality of Child Care 
(April 1997) 

This report presents the results of a 2-year study of the quality of 
child care in the 12 pioneer partnerships. 

Kindergartners’ Skills in Smart Start Counties in 1995: 

A Baseline From Which to Measure Change (July 1997) 

This report presents baseline findings of kindergartners’ skills in 
the 43 Smart Start counties. 

Child Care in the Pioneer Partnerships 1994 and 1996 
(December 1997) 

This report presents more detailed information about child care 
centers that were included in The Effects of Smart Start on the 
Quality of Child Care (April 1997). 

Families & the North Carolina Smart Start Initiative 
(December 1997) 

This report presents findings from family interviews of families who 
participated in Smart Start in the pioneer counties. The interviews 
included questions about child care, health services, family activities 
with children, and community services and involvement. 



North Carolina’s Smart Start Initiative: 
1994-95 Annual Evaluation Report 
(June 1995) 

This report summarizes the evaluation 
findings to date from both quantitative and 
qualitative data sources. 



The Effects of Smart Start Child Care on Kindergarten Entry Skills 
(June 1998) 

This report presents results from kindergartners who attended 
Smart-Start-funded child care centers compared to a random group 
of kindergartners who attended a broad range of child care or no 
child care. 



Reinventing Government? Perspectives 
on the Smart Start Implementation 
Process (November 1995) 

This report documents pioneer partner- 
ship members’ perspectives on 2 major 
process goals of Smart Start: non- 
bureaucratic decision making and broad- 
based participation. 

Center-based Child Care in the Pioneer 
Smart Start Partnerships of North 
Carolina (May 1996) 

This brief report summarizes the key 
findings from the 1994-95 data on child 
care quality. 

Effects of Smart Start on Young Children 
with Disabilities and their Families 
(December 1996) 

This report summarizes a study of the 
impact of Smart Start on children with 
disabilities. 



Effect of a Smart Start Playground Improvement Grant on Child 
Care Playground Hazards (August 1998) 

This report presents results from a comparison of the playground 
safety of child care playgrounds in a county that used Smart Start 
funds for playground improvement compared to a non-Smart Start 
county. 

Smart Start and Local Inter-Organizational Collaboration 
(August 1998) 

This report presents data about the effectiveness of the Smart Start 
initiative on improving collaborative relationships. Qualitative and 
quantitative data were obtained from 269 respondents in 10 local 
Partnerships. 

Smart Start Client Information System Feasibility Study 
(September 1998) 

This report presents findings from a study of the feasibility of 
creating a system to count uniquely all children and families served 
by Smart Start. 
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To Obtain copies of these reports, 
please call Marie Butts at; 

( 919 ) 966-4295 

or 

Email her at Butts@unc.edu 



Visit Our Webpage 
www.fpg.unc.edu/~smartstart 
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